The first related patent application for IMPLEMENTATION OF MULTI-STAGE SWITCHING NETWORKS generally concerns the design of multi-stage interconnection switching networks that provide for the exchange of data between multiple electronic devices, and more particularly concerns the logic organization and layout of semiconductor die, and the associated wiring between such die, for implementing large and very large three-dimensional multi-stage interconnection networks. The multi-stage interconnection networks so designed are characterized by (i) an efficient logical organization, (ii) a very large size that typically interconnects of the order of 4096 and more communication ports, and (iii) a sophisticated, three-dimensional, interconnection geometry.
The present application concerns the rule-directed design of three-dimensional multi-stage interconnection networks based on (i) electrical interconnection proceeding through multiple parallel wires, normally in the form of flexible ribbon cable, extending in a flat plane between (ii) multiple planes, or modules, that are preferably orthogonal to the planar interconnecting wires in which modules reside multiple standard switching chips that also constitute portions of the interconnection paths. The present application is thus related to the predecessor application as a particular methodology for realizing a three-dimensional electrical interconnectionxe2x80x94particularly between large numbers of points at high densities as epitomizes a very large multi-stage switching, or interconnection, network. Multi-stage interconnection, or switching, networks of the present invention will be characterized by the orderly, rule-directed, co-location of a greater density of interconnection wiresxe2x80x94normally in the form of flexible ribbon cablexe2x80x94laid flat and parallel, or, in regions, orthogonal, within small volumes between each of successive planes, or modules, within which reside standard interconnection routing chips, or dice. The physical geometry of such a multi-stage interconnection, or switching, network will appear dense, and complex, but regular and ordered.
The second related patent application for PAD AND CLIP GEOMETRIES FOR MOUNTING AND ELECTRICALLY CONNECTING RIBBON CABLES TO SWITCHING CHIPS IN SPACED-PARALLEL PLANAR MODULES concerns a preferred physical geometry for each of (i) a spring connector for securing flat circuit, typically flexible ribbon cable, ends in pressured contact with a substrate (of a plane, or module), in which is present (ii) a pattern of electrical pads. The electrical and mechanical connection taught within the second related patient application is characterized in that (i) flat circuits, normally in the form of flexible ribbon cables, are routed through free space substantially in each of two orthogonal planes while (ii) the points of connection to the bent ends of all such flat circuits (e.g., ribbon cables) are arrayed along diagonals in yet another, further orthogonal, plane of connection that is established by arrayed chips held in a planar module, or tile. In simple terms, the 90xc2x0 bent wire ends of one flat circuit (ribbon cable) will be connected in the orthogonal plane of a module simultaneously that the 90xc2x0 bent wire ends of another flat circuit (another ribbon cable)xe2x80x94located in another, orthogonal, planexe2x80x94are connected in the plane of the same module. Although this can clearly be done when spacing is adequate, in actual implementation the flat circuits (ribbon cables) and modules will be seen to be tightly packed. This second related patent application is thus related to the present application as a preferred means for realizing a compact electrical connector/electrical connection in a three-dimensional multi-stage interconnection, or switching, network.
1. Field of the Invention
The present invention generally concerns the design of three-dimensional multi-stage interconnection networks based on (i) many flat circuits, normally in the form of flexible ribbon cables, connecting between each of (ii) multiple planes, or modules, or tiles in which reside both standard chips and printed wiring that also constitute portions of the interconnection paths.
The present invention particularly concerns very large scale interconnection, or switching, networks that are not only logically efficient, and potentially also non-blocking, but that are, in accordance with the present invention, physically realized in structures that are each of economic, manufacturable, orderly, and maintainable because they are constructed in accordance with a rule-directed physical (inter)connection geometry. The rule-directed interconnection geometry is efficient, reliable, regular, and, arguably, elegant.
2. Description of the Prior Art
The present invention is concerned with the physical realization of multi-stage interconnection switching networks which provide for the efficient and rapid communication of data between large numbers of electronic devices, typically hundreds and even thousands of computer data processors. The multi-stage interconnection switching networks involve large numbers of semiconductor switch dice located in co-parallel planar modules which comprise the stages, and (ii) the associated electrical interconnection wiring between the planar-arrayed dice in each stage, forming thus a switching network in three dimensions.
The switching networks of the present invention are designed with switches, or switchpoints, that are located in logical rows and in logical columns, as is common. Such switching networks are commonly physically constructed with the physical switchesxe2x80x94which are commonly implemented from semiconductor dicexe2x80x94arranged into physical ranks and physical files. When large numbers of electronic devices must be interconnected by even larger numbers of switches, the switches are commonly logically and physically arrayed as multiple stages. Because laying out each of the stages on the same plane soon becomes unwieldy large, each stage is laid out on a single plane, and the planes are stacked one atop another in three dimensions.
If, for smaller switching networks, all the switches, or switchpoints, are physically located in a common planexe2x80x94such as on a single circuit panel or on a number of circuit panels adjacent to one another, then the interconnection wiring between the outputs and the inputs of the various switches of this circuit panel are clearly accomplished in, or substantially in, the plane of the circuit panel. When several circuit panels are used, it is common to connect from one to the next by edge connectors. The several printed circuit panels may be located in a single plane, and the edge connections may thus also be in this plane. However, if the edge connections are made with flexible cable, including the multi-conductor flexible cable commonly known as ribbon cable, as is common, then the panels may be arrayed spaced parallel to each other in a stack.
Although wiring has occasionally been made from central area regions of one panel directly across to corresponding central area regions of an adjacent parallel panel, at least two problems have beset making electrical connection directly from panel to panel in the volume between them so as to attempt to realize high density, and minimal communication delay, within a multi-stage switching network. If the interconnecting wires are permanently, or semi-permanently, affixed to the panels, such as by soldering in holes, then the successive panels must be xe2x80x9claid upxe2x80x9d in order during construction, and become effectively impossible to disassemble for maintenance to replace any chip switches that have failed. If the interconnecting wiresxe2x80x94commonly in the form of ribbon cables with stripped wire endsxe2x80x94are instead not to be placed through holes in the panels and soldered, then a reliable form of electrical connection, and electrical connector, is needed between the interconnecting wires and the transverse panels. Moreover, even if a suitable connector is found, the typically high wiring density between the panels tends to turn the volume between the panels into a xe2x80x9crat""s nestxe2x80x9d, with physical conflicts between wires forcing the panels to greater separation and the interconnection wires to greater lengths, and to associated communications signal delay, than would desirably be the case.
These problems have heretofore been so severe that free-space optical interconnections have been contemplated between panels of a stack for appreciably-sized multi-level switching network cross-connecting 256 nodes or more. Photons, being a type of boson, pass though each other in free space without appreciable cross-interference, and have thus been hypothesized to be more suitable for three-dimensional spatial-point to spatial-point communicative interconnection than would be electrons, which, as fermions, strongly interfere with each otherxe2x80x94whether or not carried on wires which themselves exhibit spatial impermeability. Although there may yet be a point where three-dimensional optical interconnection, with all its overhead of transformation from and to the electrical signals presently used for computation and switching, becomes useful or even dominant, the present invention will be seen to concern good xe2x80x9cold fashionedxe2x80x9d point-to-point wiring, extending such wiring into multi-level networks of sizes heretofore believed impracticable, if not impossible, of being implemented by purely electrical connection.
The present invention will shortly been seen to contemplate a truly three-dimensional switching network where communication connections are made in the volume between adjacent parallel circuit panels, and along routes between the panels which routes are transverse to the planes of the panels. In accordance with the present invention, the wiring between adjacent panels, although extremely dense and spatially sophisticated, is highly regular, and substantially devoid of conflicts. In accordance with the second related invention, regular and reliable electrical connection may be made from the flat wiring circuits, or flexible ribbon cables, located between adjacent panels and in plane transverse to the panels. Switching circuits are located in electrical dice upon, and co-planar with, the panels.
The three-dimensional switching network physical geometries of the present invention are suitable to implement diverse logical switching networks. The prior art baseline network of FIG. 1, the reverse Banyan network of FIG. 2, the Cantor network of FIG. 3, the two-planed layered network of FIG. 4, or any of the other layered networks that are described in U.S. Pat. No. 4,833,468, which is entitled LAYERED NETWORK and which issued in the names of selected inventors of the present invention on May 23, 1989, are all examples of networks which may beneficially use the three-dimensional interconnection geometries of present invention.
However, the preferred wiring rules, patterns and topology of the present inventionxe2x80x94i.e. the locations of the interconnection wires between the panel-mounted switchesxe2x80x94are based upon the logic switch design taught within the first related patent application. The logic design of the switch is not difficult to understand. Why, however, one particular logic design should be selected from among many competing candidatesxe2x80x94so as to make realizable an efficient physical geometryxe2x80x94will become increasingly clear in the present disclosure.
The invention of the related patent calls for a (i) rotating, (ii) folding and (iii) squaring process on a logical network. This (i)-(iii) process substantially reduces the length of (wired) connections between, as ultimately physically laid out, the physical switches within the physical network. In other words, the logical network is selectively xe2x80x9cpre-conditionedxe2x80x9d by moving the logical switching elements around (totally without change to the logical function of these elements, nor of the multi-level switching network that they serve to implement) so as to make more efficientxe2x80x94even possiblexe2x80x94the (wired) implementation of the physical switching network which is the concern of the present invention.
Moreover, the approach of the related invention will be seen to permit the construction of very large networks by combining smaller networks. In networks constructed with the rule-directed geometries of the present invention, the length of the longest connections between switches will be found to be proportional to the square root of the number of ports provided by the network.
Furthermore, once the power of this approach (i.e., xe2x80x9cpre-conditioningxe2x80x9d a logical multi-stage switching network so as to then best permit subsequently physically realizing a real-world multi-stage switching network by the rule directed geometries of the present invention) is fully understood, it will be realized that the approach is of general applicability to implementing multi-stage switching networks.
In particular, the layered network of the U.S. Pat. No. 4,833,468 has, as a design, a number of desirable features. However, the layered network design of this patent was initially implemented in accordance with precise algorithms so that each network load size required a unique wiring topology and various different kinds of switching panels. The present invention will show how to physically implement, by way of example, a layered network in a regular, scalable, form. In other words, the same parts may be used to build very large, and stupendously large, switching networks as are used to build large switching networks. Although a layered network from the former patent is used as an example to illustrate the present and related inventions, the present invention may be applied to the various other types of multi-stage switching networks, including (i) baseline, (ii) Banyan and (iii) Cantor networks, (iv) all the layered networks that may be constructed in accordance with the teachings of U.S. Pat. No. 4,833,468, and (v) still other types of switching networks including those of the above-referenced related patent applications.
The present invention contemplates a particular physical realization of multi-stage three-dimensional switching networks, especially as may be implemented at large and very large scale, for electrically communicatively interconnecting large numbers of electrically communicating devices, normally hundreds and thousands of such devices. The physical realization transpires by use of (i) multi-conductor flat circuits, normally in the form of flexible ribbon cable, to connect (ii) multiple planar panels, or modules, or tiles, in each of which resides both some number of standard switching chips, and printed wiring between the chips, that also constitute a portion of the interconnection paths.
The (ii) panels, or modules, or tiles and their contained printed circuit wiring and switching chips are the xe2x80x9clayersxe2x80x9d of a logical multi-layer switching network. (Actually two network switch layersxe2x80x94logically separated yet!xe2x80x94are sometimes within the same panel.) The (i) multi-conductor flat circuits, or ribbon cable, form the electrical connection between the xe2x80x9clayersxe2x80x9d of the logical multi-layer switching network.
Thus the physical multi-layer switching network includes (a) a number of spaced-parallel planar panels electrically interconnected by (b) substantially-planar multi-conductor circuits, or cables, in three-dimensional space. More particularly, a plurality N1 of the (a) spaced-parallel planar panels are stacked from a bottom panel (a xe2x80x9clayerxe2x80x9d of the logical switching network) that is located physically closest to the devices communicatively interconnected by the switching network to a top panel (another xe2x80x9clayerxe2x80x9d of the logical switching network) that is located at the furthest distance from the communicatively interconnected devices. Already this geometry is somewhat strange: it represents a logical switching network that has been folded. Namely, signals originating from communicatively interconnected devices go out onto the switching network at a nearest panel and, after transiting panel-to-panel all the way to the most distant panel, then wrap back through the same stacked panels until reaching their destinations among (typically different ones of) the same devices. The signals are normally self-routing, with all necessary switching and routing information being carried in the signal.
Each (a) spaced-parallel planar panel mounts (a1) a number N2 of multi-chip modules known as tiles. Each tile has (a1a) a number N3 of switchpoints. Each of the switchpoints is realized as (a1a1) a number N4 of dice, having pads and held to the tiles, for switching received electrical signals, (a1a2) a number of electrically conductive vias through the tiles, (a1a3) a number of N5 of electrically conductive pads upon both sides of the tiles, and (a1a4) wiring layers upon the tile electrically connecting the dice pads, the vias, and the electrically conductive pads. The electrically-connectable pads are presented upon both sides of the tiles, and of the panels in which the tiles are mounted.
The (b) flexible substantially-planar multi-conductor cables are located between adjacent panels. These cables serve to electrically connect the pads on one panel to the pads of a facing surface of an adjacent panel. These cables make a number N6 of flex connections total.
So far the constructionxe2x80x94while certainly specific in the described (a) planar panels, (a1) multi-chip modules, or tiles, (a1a) switchpoints, (a1a1) dice, (a1a2) vias, (a1a3) electrically conductive pads, and (a1a4) wiring layersxe2x80x94has been somewhat arbitrary. In other words, if a three-dimensional multi-level switching network was to be contemplated in the first instance, it might reasonably be assumed that things like panels and switches and cables would need be connected together, or necessity, in some kind of structure, and in some kind of order. However, herein next lies a unique characteristic of the three-dimensional multi-stage switching network in accordance with the present invention: the (b) substantially-planar multi-conductor cables are aligned in their extension between the (a) spaced parallel planar panels very substantially in but two sets of parallel planes, the planes of each which set are orthogonal to the planes of the other. Both the two sets of planes are also orthogonal to the planes of the panels, which constitute a third set of planes.
These orthogonal planes in which planar multi-conductor, or ribbon, cables are routed will prove to be readily visibly detectable in the drawings. The reason that this orthogonal routing is significant is that xe2x80x9cflexible cablexe2x80x9d can twist and turn every which wayxe2x80x94exactly as it is most often called upon to do. But consider the quite opposite, and rigid, geometry of the present invention: all the electrically conductive interconnectionsxe2x80x94whether wires in cables or printed circuit wiring traces on rigid panelsxe2x80x94reside in some particular plane in one of three sets of orthogonal planes. The cables between the panels are located in very substantially in but two sets of spaced-parallel planes. The planes of each set are orthogonal to the planes of the other set. The panels occupy a third set of spaced-parallel planes. The planes of this third set are orthogonal to all the planes of both of the other two sets. Roughly speaking, everything is xe2x80x9csquarexe2x80x9d to everything else, and so appears. For this reason, the common name of a multi-level switching network constructed in accordance with the present and related inventions is a xe2x80x9cfour square networkxe2x80x9d, and the network will be so called in the present application.
In further detail, the planar panels preferably include channels within which coolant flows for removal of heat developed in the (physical) dice (containing the logic switchpoints) upon the tiles that are mounted to the panels. Because the electronic dice consume power, the planar panels normally further include power and ground connections to the tiles on which reside the dice.
Considerably more arbitrarily, each of the multi-chip modules known as tiles preferably has and defines recesses in which are located the several dice that serve to realize a number of logic switchpoints. The preferred mounting of the dice is recessed so that one surface of each dice is substantially even with a surface of the tile, thereupon permitting that wiring upon the tile may be implemented as printed circuit traces substantially in a single plane upon the exposed surfaces of both the tiles and the dice.
The flexible substantially planar multi-conductor cable preferably consists of ribbon cable with stripped wire ends bent ninety degrees so that these wire ends may lie flush against the pads of the panels. The preferred ribbon cable is treated, commonly by heat, at its stripped ends so as to become rigid.
However, and in full accordance with the dictate of the present invention that these cables should lie strictly in two sets of (orthogonal) planes, the cables may alternatively be implemented as rigid circuits. Rigidity in the area of the panel-interconnecting cables is normally neither required nor desired. However, if a (physical) multi-layer switching network of the present invention must be made rigidly strong, such as for being launched into space, it may readily be so made.
Connectors of a unique design forcibly pressure the stripped wire ends of the preferred ribbon cables against the pads of the panels so as to make electrical connection thereto. The pads on the panels are preferably located in a pattern that is identical upon both sides of the panels. The connection of (the stripped, bent) wire ends of each ribbon cable is preferably at a series of pads arrayed along a line that isxe2x80x94quite curiously relative to both (i) the normal layout of pad connections to electronic devices, and (ii) the prevailing square geometry of the present three-dimensional multi-stage switching networkxe2x80x94at a diagonal to the sides of the panel.
The multi-stage switching may be built by way of example, with switchpoints that are 3xc3x973, meaning that three signals in are selectively controllably routed to three signals out. In such a case, and in accordance with the expandability of the scheme of the present invention, many different sizes and configurations of the three-dimensional multi-stage switching network in accordance with the present invention are possible. Three such sizes and configurations are given in the following table.
Note that the number of electrically conductive pads numbers 16, or the same as the number of switchpoints, upon each of the two sides of the tiles.
There may optionally be an additional direct, electrical connection, preferably still via a multi-conductor cable, between each tile and a facing tile upon an adjacent panel. In that case the number N6 of flex connections increases to 2064+64=2128 in total for the cases of both the 256-device and of the 1024-device communicatively-interconnecting multi-stage switching-device networks, and to 49152+1536=50688 in total for the 4096 device communicatively-interconnecting multi-stage switching network.
In each of the configurations, a panel is preferably 64 inches or less on each side, and the space between adjacent panels is preferably 4 inches or less. Accordingly, the longest flex connection between any two adjacent panels that is made by any of the flexible substantially planar multi-conductor cables is no more than 38 inches, or one half a panel""s 64 inch dimension plus the 4 inch distance between adjacent panels.
As previously mentioned, the switchpoints are preferably self-routing, meaning that the connection of electrical signals from switchpoint input to switchpoint output is determined by data received at the switchpoint input.